Papers by Alan W Black
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Combining visual modality with pretrained language models has been effective for descriptive tasks such as image captioning. |
| Approach: | They ask: do multimodal models combine visual and visual adapted language models? they find that CLIP image representations and scaling of language models do not consistently improve self-rationalization in multimodal tasks. |
| Outcome: | The proposed model types do not consistently improve self-rationalization in multimodal tasks. |
Topological Sort for Sentence Ordering (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work on sentence ordering task has framed it as a sequence prediction problem. |
| Approach: | They propose a new constraint solving problem and propose 'human evaluation' they propose to capture coherence in documents by arranging sentences in the correct order . |
| Outcome: | The proposed technique captures coherence in documents better than previous approaches. |
Top-Down Structurally-Constrained Neural Response Generation with Lexicalized Probabilistic Context-Free Grammar (N19-1)
Copied to clipboard
| Challenge: | Neural encoder-decoder architectures have shown promise for natural language generation. |
| Approach: | They propose to generate words according to order of first appearance in lexicalized PCFG parse tree . they also combine neural model with symbolic approach to generate syntactic structure . |
| Outcome: | The proposed method improves over sequence-to-sequence baseline in diversity and relevance. |
What A Sunny Day ☔: Toward Emoji-Sensitive Irony Detection (D19-55)
Copied to clipboard
| Challenge: | Existing datasets for irony detection only contain 10% of ironic tweets with emojis . 45% of internet users in the united states use an e-moji in social media . |
| Approach: | They propose to use emojis to analyze irony detection datasets to train classifiers. |
| Outcome: | The proposed pipeline can be used to analyze irony detection datasets using emojis. |
Multimodal, Multilingual Grapheme-to-Phoneme Conversion for Low-Resource Languages (D19-61)
Copied to clipboard
| Challenge: | Grapheme-to-phoneme conversion (g2p) is a task of predicting the pronunciation of words from their orthographic representation. |
| Approach: | They propose to leverage audio data as an auxiliary modality in a multi-task training process to learn a more optimal grapheme representation. |
| Outcome: | The proposed model reduces phoneme error rate to 2.46% on in-domain test set compared to unimodal spelling- pronunciation model. |
Reading Between the Lines: Exploring Infilling in Visual Narratives (2020.emnlp-main)
Copied to clipboard
| Challenge: | Generating long form narratives from multiple modalities requires a model to learn surrounding contextual information by masking spans of input while decoding attempts in generating the entire text. |
| Approach: | They propose to use infilling techniques to generate textual descriptions from images that are rich in contextual dependencies. |
| Outcome: | The proposed model outperforms existing models in visual storytelling by generating text from a large scale dataset of 46,200 procedures and 340k pairwise images and textual descriptions. |
Focused Attention Improves Document-Grounded Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document grounded generation is the task of using the information provided in a document to improve text generation. |
| Approach: | They propose two new document grounded generation tasks that use information provided in a document to improve text generation. |
| Outcome: | The proposed models outperform existing methods on automated and human evaluation for closeness to reference and relevance to the document. |
Phone Features Improve Speech Translation (2020.acl-main)
Copied to clipboard
| Challenge: | End-to-end models for speech translation more tightly couple speech recognition (ASR) and machine translation (MT) compared to cascades, but performance gap remains in low-resource conditions . |
| Approach: | They propose two methods to incorporate phone features into current neural speech translation models. |
| Outcome: | The proposed models outperform existing models and cascades by up to 9 BLEU on low-resource conditions. |
ClarQ: A large-scale and diverse dataset for Clarification Question Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing datasets hinder development of large-scale models capable of generating and utilising clarification questions. |
| Approach: | They propose a bootstrapping framework that utilises a neural network architecture to classify clarification questions based on post-comment tuples extracted from stackexchange. |
| Outcome: | The proposed framework aims to increase the accuracy of the classifier and increase recall of clarification questions by applying it to question-answering tasks. |
Storyboarding of Recipes: Grounded Contextual Generation (P19-1)
Copied to clipboard
| Challenge: | Using a dataset for sequential procedural (how-to) text generation from images, we show that 61% of the users found our proposed model is better than the baseline model in terms of overall recipes. |
| Approach: | They propose a dataset for sequential procedural (how-to) text generation from images in cooking domain. |
| Outcome: | The proposed model achieves a METEOR score of 0.31, an improvement of 0.6 over the baseline model. |
Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation (P19-1)
Copied to clipboard
| Challenge: | Previous work on end-to-end translation from speech uses frame-level features as speech representations, which creates longer, sparser sequences than text. |
| Approach: | They propose a method to generate compressed phoneme-like speech representations that generate shorter, higher-level source sequences for translation. |
| Outcome: | The proposed method improves translation performance by 5 BLEU on high and low resource languages and reduces training time by 60%. |
Question Answering for Privacy Policies: Combining Computational and Legal Perspectives (D19-1)
Copied to clipboard
| Challenge: | Privacy policies are long and complex documents that are difficult for users to read and understand. |
| Approach: | They present a corpus of 1750 questions about privacy policies of mobile applications and over 3500 expert annotations of relevant answers. |
| Outcome: | The proposed corpus of 1750 questions on privacy policies shows that a strong neural baseline underperforms human performance by almost 0.3 F1 on PrivacyQA. |
Grounding ‘Grounding’ in NLP (2021.findings-acl)
Copied to clipboard
| Challenge: | Cognitive Science defines "grounding" as the process of establishing mutual information between two interlocutors. |
| Approach: | They examine the gaps between NLP and Cognitive Science definitions of "grounding" they propose ways to create new tasks or repurpose existing ones to achieve a more complete sense of grounding . |
| Outcome: | The authors examine the gaps between definitions of grounding and cognitive science . they show that there are ways to improve existing tasks or repurpose existing ones . |
Style Transfer Through Back-Translation (P18-1)
Copied to clipboard
| Challenge: | a new method for automatic style transfer is proposed to preserve the meaning of the text while reducing stylistic properties. |
| Approach: | They propose a method for automatic style transfer that uses latent representations of the input sentence to preserve meaning while reducing stylistic properties. |
| Outcome: | The proposed method improves on sentiment, gender and political slant styles on three different styles. |
A Corpus for Large-Scale Phonetic Typology (2020.acl-main)
Copied to clipboard
Elizabeth Salesky, Eleanor Chodroff, Tiago Pimentel, Matthew Wiesner, Ryan Cotterell, Alan W Black, Jason Eisner
| Challenge: | Existing multilingual speech corpora have limited data in many languages . existing corpus is limited to a small number of languages with available data . |
| Approach: | They propose a large-scale phonetic typology corpus with phoneme-level labels and phoneme alignments in 690 readings spanning 635 languages. |
| Outcome: | The proposed corpus covers 635 languages and includes acoustic-phonetic measures of vowels and sibilants. |
CTC Alignments Improve Autoregressive Translation (2023.eacl-main)
Copied to clipboard
Brian Yan, Siddharth Dalmia, Yosuke Higuchi, Graham Neubig, Florian Metze, Alan W Black, Shinji Watanabe
| Challenge: | Connectionist Temporal Classification (CTC) is widely used for automatic speech recognition (ASR) but lags behind attentional decoder approaches in terms of translation quality. |
| Approach: | They propose to use a CTC/attention framework to validate this hypothesis by modifying the Hybrid CTC-Attention model proposed for automatic speech recognition to support text-to-text translation (MT) and speech-totext translation. |
| Outcome: | The proposed model outperforms pure-attention baselines across six translation tasks. |
Exploring Controllable Text Generation Techniques (2020.coling-main)
Copied to clipboard
| Challenge: | Neural controllable text generation has a plethora of applications but there is no unifying theme. |
| Approach: | They propose a new schema for the control of attributes in the generation process by classifying it into five modules and providing an analysis on the advantages and disadvantages of these techniques. |
| Outcome: | The proposed frameworks can be used to control the attributes of natural sentences and to modulate the formality and politeness of emails. |
Boosting Dialog Response Generation (P19-1)
Copied to clipboard
| Challenge: | Neural models generate the most common and generic responses all the time . Empirical results show that our method can significantly improve the diversity of responses generated by sequence-to-sequence models. |
| Approach: | They propose an iterative training process and ensemble method based on boosting to improve the diversity of responses generated by neural models. |
| Outcome: | Empirical results show that the proposed method significantly improves diversity and relevance of responses generated by all models. |
Breaking Down Walls of Text: How Can NLP Benefit Consumer Privacy? (2021.acl-long)
Copied to clipboard
| Challenge: | Privacy policies are long and complex documents that are difficult for users to read and comprehend. |
| Approach: | They propose language technologies to help users reclaim control over their privacy . they highlight many remaining opportunities to develop more precise or nuanced language technologies . |
| Outcome: | The proposed language technologies can address the privacy information gap . they can be more precise or nuanced in the way they use the text of privacy policies. |
Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings (N19-1)
Copied to clipboard
| Challenge: | Existing methods to debias word embeddings in binary settings such as gender and religion are limited to binary labels, whereas word2vec embedders can be used to propagate biases. |
| Approach: | They propose a method to debias word embeddings in multiclass settings such as gender and religion, extending the work of Bolukbasi et al. (2016). |
| Outcome: | The proposed method maintains the efficacy in standard NLP tasks while maintaining the utility of embeddings. |
Phone Inventories and Recognition for Every Language (2022.lrec-1)
Copied to clipboard
| Challenge: | Identifying phone inventories is crucial component in language documentation and preservation of endangered languages. |
| Approach: | They propose a probabilistic and non-probabilistic phone inventory model that estimates the phone inventory for any language listed in Glottolog. |
| Outcome: | The proposed model outperforms baseline models by 6.5 F1 and improves the PER (phone error rate) in phone recognition by 25%. |
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Question-Answering (QA) systems are deployed in the real world . a lack of research attention has been devoted to studying the issues that arise when people use QA systems. |
| Approach: | They show that component components that precede an answering engine can introduce varied and considerable sources of error. |
| Outcome: | The proposed evaluations highlight the need for QA evaluation to expand to consider real-world use. |
A Dataset for Document Grounded Conversations (D18-1)
Copied to clipboard
| Challenge: | a dataset of document grounded conversations provides information on content of a document . current datasets lacking conversation grounding do not provide this information . |
| Approach: | They propose a document grounded dataset for conversations . they use Wikipedia articles about popular movies to define document grounded conversations based on their results . |
| Outcome: | The proposed dataset provides a source of information and provides benchmark performance on the task of generating the next response. |
Politeness Transfer: A Tag and Generate Approach (2020.acl-main)
Copied to clipboard
Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W Black, Shrimai Prabhumoye
| Challenge: | Prior work on text style transfer has not focused on politeness as a style transfer task and we argue that defining it is cumbersome. |
| Approach: | They propose a task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning. |
| Outcome: | The proposed model outperforms state-of-the-art methods on content preservation and style transfer accuracy. |
Formality Style Transfer for Noisy, User-generated Conversations: Extracting Labeled, Parallel Data from Unlabeled Corpora (D19-55)
Copied to clipboard
| Challenge: | Typical datasets used for style transfer in NLP contain aligned pairs of two opposite extremes of a style. |
| Approach: | They propose a technique to derive a dataset of aligned pairs from an unlabeled corpus by using an auxiliary dataset, allowing for in-domain training. |
| Outcome: | The proposed method significantly outperforms OpenNMT’s Seq2Seq model trained on the Yahoo Formality Dataset and 6 novel datasets. |
A Resource for Computational Experiments on Mapudungun (2020.lrec-1)
Copied to clipboard
Mingjun Duan, Carlos Fasola, Sai Krishna Rallabandi, Rodolfo Vega, Antonios Anastasopoulos, Lori Levin, Alan W Black
| Challenge: | Low-resource languages still lag behind in documenting endangered languages . a large corpus of culturally significant conversations is available for computational experiments . |
| Approach: | They propose a resource for computational experiments on Mapudungun, a polysynthetic indigenous language spoken in Chile. |
| Outcome: | The proposed corpus provides 142 hours of culturally significant conversations in Mapudungun . the language is spoken by the Mapuche people of southern Chile and western argentina . |
Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switching (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Code-switching (CS) is a phenomenon of switching between multiple languages . current models cannot handle CS due to lack of annotated data and limited resources. |
| Approach: | They propose a self-training method to repurpose existing models using a switch-point bias by leveraging unannotated data to reduce the gap between the switch point performance and retain overall performance on two distinct language pairs. |
| Outcome: | The proposed model reduces the gap between the switch point performance while retaining the overall performance on two distinct language pairs. |
Learning to Order Graph Elements with Application to Multilingual Surface Realization (D19-63)
Copied to clipboard
| Challenge: | Recent advances in deep learning have shown promises in solving combinatorial optimization problems, such as sorting variable-sized sequences. |
| Approach: | They propose an encoder-decoder framework that learns the representation for each element and predicts the ordering of each local neighborhood of the graph in turn. |
| Outcome: | The proposed framework outperforms previous frameworks on multilingual surface realization tasks while outperforming those below by a large margin. |
AlloVera: A Multilingual Allophone Database (2020.lrec-1)
Copied to clipboard
David R. Mortensen, Xinjian Li, Patrick Littell, Alexis Michaud, Shruti Rijhwani, Antonios Anastasopoulos, Alan W Black, Florian Metze, Graham Neubig
| Challenge: | Phonemes are contrastive phonological units, and allophones are their various concrete realizations. |
| Approach: | They propose a resource that maps allophones to phonemes for 14 languages . they propose phonological representations that are much closer to a universal transcription . |
| Outcome: | The proposed resource maps from 218 allophones to phonemes for 14 languages. |
Case Study: Deontological Ethics in NLP (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work in natural language processing (NLP) has focused on ethical challenges . ethical foundations of NLP systems have not been explored . |
| Approach: | They propose to use deontological ethics to analyze ethical issues in natural language processing from the perspective of NLP. |
| Outcome: | The proposed ethical frameworks are based on the generalization principle and respect for autonomy through informed consent. |
Token-level Sequence Labeling for Spoken Language Understanding using Compositional End-to-End Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | End-to-end spoken language understanding systems model sequence labeling as a sequence prediction task causing a divergence from its well-established token-level tagging formulation. |
| Approach: | They propose to model sequence labeling as a sequence prediction task . their systems explicitly separate the added complexity of recognizing spoken mentions from the NLU task of sequence labelling . |
| Outcome: | The proposed systems outperform both cascaded and direct models on a labeling task of named entity recognition across SLU benchmarks. |